{
 "cells": [
        {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table align=\"left\">\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/gimseng/99-ML-Learning-Projects/blob/master/005/solution/sentiment_analysis.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import random\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n",
    "from sklearn.metrics import f1_score\n",
    "\n",
    "import json, urllib.request"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Data Class"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our first model will be automatically classifying positive and negative comments"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Creating data to train the models is not an good approach, getting data by some other sources or by web crawling is one the best techniques, for negative and positive sentence data you can craw the amazon's review column of any product."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "I bought both boxed sets, books 1-5.  Really a great series!  Start book 1 three weeks ago and just finished book 5.  Sloane Monroe is a great character and being able to follow her through both private life and her PI life gets a reader very involved!  Although clues may be right in front of the reader, there are twists and turns that keep one guessing until the last page!  These are books you won't be disappointed with.\n5.0\n"
    }
   ],
   "source": [
    "# Storing the Path of file in a variable\n",
    "project_url = 'https://raw.githubusercontent.com/gimseng/99-ML-Learning-Projects/master/005/'\n",
    "file_name = project_url+'data/books_small_10000.json'\n",
    "\n",
    "# Opening JSON file and reading it line by line.\n",
    "with urllib.request.urlopen(file_name) as f:\n",
    "    for line in f:\n",
    "        review = json.loads(line)\n",
    "        # Getting review text\n",
    "        print(review['reviewText'])\n",
    "        # Getting the Overall rating\n",
    "        print(review['overall'])\n",
    "        break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Storing the Path of file in a variable\n",
    "file_name = project_url+'data/books_small_10000.json'\n",
    "\n",
    "# Create empty list to store tuple objects of every data\n",
    "reviews = []\n",
    "\n",
    "# Opening JSON file and reading it line by line.\n",
    "with urllib.request.urlopen(file_name) as f:\n",
    "    for line in f:\n",
    "        review = json.loads(line)\n",
    "        reviews.append((review['reviewText'], review['overall']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.0"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Printing Random object from the reviews\n",
    "reviews[5]\n",
    "reviews[5][1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now creating a Data class for our reviews and we are gonn initialise it with text and score\n",
    "# Now insted of appending this tuple we will create a Review object and pass in text and score.\n",
    "\n",
    "class Review:\n",
    "    def __init__(self, text, score):\n",
    "        self.text = text\n",
    "        self.score = score\n",
    "        \n",
    "\n",
    "        # Storing the Path of file in a variable\n",
    "file_name = project_url+'data/books_small_10000.json'\n",
    "\n",
    "# Create empty list to store tuple objects of every data\n",
    "reviews = []\n",
    "\n",
    "# Opening JSON file and reading it line by line.\n",
    "with urllib.request.urlopen(file_name) as f:\n",
    "    for line in f:\n",
    "        review = json.loads(line)\n",
    "        reviews.append(Review(review['reviewText'], review['overall']))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.0"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Getting score \n",
    "reviews[5].score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "'I hoped for Mia to have some peace in this book, but her story is so real and raw.  Broken World was so touching and emotional because you go from Mia\\'s trauma to her trying to cope.  I love the way the story displays how there is no \"just bouncing back\" from being sexually assaulted.  Mia showed us how those demons come for you every day and how sometimes they best you. I was so in the moment with Broken World and hurt with Mia because she was surrounded by people but so alone and I understood her feelings.  I found myself wishing I could give her some of my courage and strength or even just to be there for her.  Thank you Lizzy for putting a great character\\'s voice on a strong subject and making it so that other peoples story may be heard through Mia\\'s.'"
     },
     "metadata": {},
     "execution_count": 6
    }
   ],
   "source": [
    "reviews[5].text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "\n",
    "class Sentiment:\n",
    "    NEGATIVE = 'NEGATIVE'\n",
    "    NEUTRAL = 'NEUTRAL'\n",
    "    POSITIVE = 'POSITIVE'\n",
    "\n",
    "class Review:\n",
    "    def __init__(self, text, score):\n",
    "        self.text = text\n",
    "        self.score = score\n",
    "        # initialising sentiments 4/5 stars means +ve and 1/2 stars means -ve\n",
    "        self.sentiments = self.get_sentiments()\n",
    "        \n",
    "    def get_sentiments(self):\n",
    "        if self.score <= 2:\n",
    "            return Sentiment.NEGATIVE\n",
    "        elif self.score >= 4:\n",
    "            return Sentiment.POSITIVE\n",
    "        elif self.score == 3:\n",
    "            return Sentiment.NEUTRAL\n",
    "        \n",
    "\n",
    "class ReviewContainer:\n",
    "    def __init__(self, reviews):\n",
    "        self.reviews = reviews\n",
    "        \n",
    "    def get_text(self):\n",
    "        [x.text for x in training]\n",
    "        \n",
    "    def get_sentiment(self):\n",
    "        [x.sentiment for x in self.sentiment]\n",
    "    \n",
    "    def evenly_distribute(self):\n",
    "        negative = list(filter(lambda x: x.sentiments == Sentiment.NEGATIVE, self.reviews))\n",
    "# Its looking all the reviews mapping every sentiment, its basically filtering based upon negative sentiments, keeping \n",
    "# track of that in the negative list\n",
    "        positive = list(filter(lambda x: x.sentiments == Sentiment.POSITIVE, self.reviews))\n",
    "# Distribute evenly in the prep Data cell 10.\n",
    "        positive_shrunk = positive[:len(negative)]\n",
    "        self.reviews = negative + positive_shrunk\n",
    "        # Shuffle so u wont know what comes when\n",
    "        random.shuffle(self.reviews)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Storing the Path of file in a variable\n",
    "file_name =project_url+ 'data/books_small_10000.json'\n",
    "\n",
    "# Create empty list to store tuple objects of every data\n",
    "reviews = []\n",
    "\n",
    "# Opening JSON file and reading it line by line.\n",
    "with urllib.request.urlopen(file_name) as f:\n",
    "    for line in f:\n",
    "        review = json.loads(line)\n",
    "        reviews.append(Review(review['reviewText'], review['overall']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'POSITIVE'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews[5].sentiments"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now the Machine learning algorithms loves the numerical data and its kinda hard to work with the strings,So what we are gonna doing here, we will be using count vectoriser here and break down the sentence in a dictionary\n",
    "\n",
    "like we have two sentences,\n",
    "1. This book is great !\n",
    "2. This book was so bad.\n",
    "\n",
    "So the dictionary of the words will include This, book, is, great, was, so, bad.\n",
    "so we will map these dict with the sentences itself to see what words does a sentence have so\n",
    "\n",
    "                            This book is great was so bad\n",
    "    1. This book is great !  1 |  1 | 1 |   1 | 0 |0 | 0|\n",
    "    2. This book was so bad  1 |  1 | 0 |   0 | 1 |1 | 1|\n",
    "    3. Was a great book      0 |  1 | 0 |   1 | 1 |0 | 0|\n",
    "\n",
    "\n",
    "so 1 means sentence have that word and 0 means sentence doesnt have that word, 3 rd sentence is that sentence we have never seen before but we can also map that using the knowlegede of previous words in the dictionary but we cant handle 'a' here because that is not included in the dictionary during the training time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Using train test split to split data into test and training\n",
    "# What list you are passing here you will get 2 times of that\n",
    "\n",
    "training, test = train_test_split(reviews, test_size = 0.33, random_state = 42)\n",
    "\n",
    "cont = ReviewContainer(training)\n",
    "# We will use evenly distribute method.\n",
    "\n",
    "cont.evenly_distribute()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6700"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(training)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now we build a classifier on that training set and after that we will test everything on our test data\n",
    "# We will have to pass our tarining data into the vectorizer, as we have to take text and predict if it is +ve or -ve \n",
    "# So what we are gonna pass into vectorizer is X which is our sentence and y = sentiments corresponding to that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_x = [x.text for x in training]\n",
    "train_y = [x.sentiments for x in training]\n",
    "\n",
    "\n",
    "test_x = [x.text for x in test]\n",
    "test_y = [x.sentiments for x in test]\n",
    "\n",
    "# We are getting the same text again\n",
    "# train_x[0]\n",
    "# train_y[0]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bag of Words Vectorization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to perform machine learning on text documents, we first need to turn the content into numerical feature vectors "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "vectorizer = CountVectorizer()\n",
    "\n",
    "# Transforming String Data to numerical data.\n",
    "# Now this is the main data we want to use while training.\n",
    "train_x_vectors = vectorizer.fit_transform(train_x) # These are 2 steps fit and transform, we can also do them individually.\n",
    "test_x_vectors = vectorizer.transform(test_x) # We just wanna transform the test data not to fit that.\n",
    "\n",
    "\n",
    "# So now our main data will be train_x_vectors and train_y and we wanna fit our data around these."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Classification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Linear SVM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import svm\n",
    "\n",
    "clf_svm = svm.SVC(kernel = 'linear')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['POSITIVE'], dtype='<U8')"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Fitting the data.\n",
    "clf_svm.fit(train_x_vectors, train_y)\n",
    "\n",
    "# Prediction using testing data\n",
    "\n",
    "clf_svm.predict(test_x_vectors[0])\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "was sent an Arc of this book for an honest review and here it is = This is the kind of book that you want to read while sitting in front of the fire with a cup of hot apple cider and a blanket over your legs.I have read many of Jaci Burton's books and have never been disappointed. This first book in her new Hope series does not disappoint either.This is the story of Emma, a new vet who has come back home to open her own practice and Luke McCormack, a police officer in the same town.Both have been previously burned by love so both have issues but, that doesn't stop them from acting on that attraction.This book pulls you in from the first page, wraps you up and doesn't let you go until the end.I loved it!\n"
     ]
    }
   ],
   "source": [
    "# If we read this test data which we have predicted we will truely identify it as positive, so our data is fitted pretty well\n",
    "print(test_x[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier()"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "clf_dec = DecisionTreeClassifier()\n",
    "\n",
    "# fitting the same exact data we did earlier\n",
    "clf_dec.fit(train_x_vectors, train_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['POSITIVE'], dtype='<U8')"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf_dec.predict(test_x_vectors[0])\n",
    "# It is working perfect"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Naive Bayes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GaussianNB()"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.naive_bayes import GaussianNB\n",
    "\n",
    "clf_gnb = GaussianNB()\n",
    "\n",
    "trainingset = np.array(train_x_vectors)\n",
    "\n",
    "clf_gnb.fit(train_x_vectors.todense(), train_y)\n",
    "\n",
    "# clf_gnb.predict(test_x_vectors[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(max_iter=500)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "clf_lr = LogisticRegression(max_iter = 500)\n",
    "\n",
    "clf_lr.fit(train_x_vectors, train_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['POSITIVE'], dtype='<U8')"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf_lr.predict(test_x_vectors[0])\n",
    "# This time it is also fitted well"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8124242424242424"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf_svm.score(test_x_vectors, test_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.843030303030303"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf_lr.score(test_x_vectors, test_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.769090909090909"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf_dec.score(test_x_vectors, test_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6587878787878788"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf_gnb.score(test_x_vectors.toarray(), test_y)\n",
    "# toarray() is used to pack the sparse matrix into dense one."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These all were the mean accuracies of our models but now we will have a look upon the f1 score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.90738061 0.40268456 0.2656    ]\n",
      "[0.92315808 0.42016807 0.29496403]\n",
      "[0.87511135 0.16368286 0.16778523]\n"
     ]
    }
   ],
   "source": [
    "# F1_scores\n",
    "# Each individual class will gets its own f1 score\n",
    "# so what does these means , lets pass in the labels\n",
    "print(f1_score(test_y, clf_svm.predict(test_x_vectors), average = None, \n",
    "      labels = [Sentiment.POSITIVE, Sentiment.NEGATIVE ,Sentiment.NEUTRAL]))\n",
    "print(f1_score(test_y, clf_lr.predict(test_x_vectors), average = None, \n",
    "      labels = [Sentiment.POSITIVE, Sentiment.NEGATIVE ,Sentiment.NEUTRAL]))\n",
    "print(f1_score(test_y, clf_dec.predict(test_x_vectors), average = None, \n",
    "      labels = [Sentiment.POSITIVE, Sentiment.NEGATIVE ,Sentiment.NEUTRAL]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So the svm is predicting POSITIVE 90% and negative for 40% time correctly and similarly we can see others performing, so we can conclude that best classifier would be the logistic regression here, now we will work on how we wanna improve f1 scores for NEGATIVE Sentiment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5611\n",
      "436\n"
     ]
    }
   ],
   "source": [
    "print(train_y.count(Sentiment.POSITIVE)) # No. of positive responses in the data\n",
    "\n",
    "print(train_y.count(Sentiment.NEGATIVE)) # No. of negative responses in the data\n",
    "\n",
    "# So due to having many positive responses our model is baised around the positives."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will add one more container class above which will evenly distribute the positives and negatives in cell 8."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2-final"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
